Export LLVM symbols for autodiff on Apple#151243
Export LLVM symbols for autodiff on Apple#151243sgasho wants to merge 3 commits intorust-lang:mainfrom
Conversation
|
This seems like it has the same goal as #150992 except implemented in a hackier way. |
|
Thanks for informing! |
|
After #150992 has been merged, I'll modify this PR or create a new PR to export LLVM symobls for autodiff on Apple target I leave this PR as a draft for now |
|
Also @Kobzol , just for awareness. |
|
Is there any other potential perf. impact other than making the toolchain larger? |
|
I don't expect it to have any relevant runtime perf impact, but given the surface of LLVM and the complexity of our bootstrap (e.g. PGO) I also wouldn't be too surprised if there's a corner case where this affects a benchmark. On Linux I would have started a perf run, but since we don't have those for apple I'd just merge it when ready. I mainly wanted to make sure that you're aware and ok with this change. |
|
Ok. Go ahead :) |
|
Ok, now that the Enzyme update landed, can you create some new linux+apple artifacts based on the latest EnzymeAD/Enzyme + #150992 (as it seems almost ready to land)? |
|
Why do we actually statically link LLVM on macOS in the first place? I know we do it on Windows to work around the limit around how much symbols can be exported from a single dll, but that is not an issue on macOS. |
|
@Kobzol might know |
|
Nope, no idea 😆 |
|
I think we currently just always use static linking, except the cases where we went out of the way to use dynamic linking for specific reasons (e.g. x86_64-unknown-linux-gnu so we can built libLLVM with LTO, PGO etc). It would probably make sense to invert this and always use dynamic linking unless there is a reason not to (Windows). Of course, dynamically linking LLVM does increase startup overhead. |
Export LLVM symbols from
rustc_driveron Apple when autodiff is enabled.On Apple targets, Enzyme can hang if rustc and Enzyme each embed their own LLVM. Exporting LLVM symbols from
rustc_driverlets Enzyme reusedependencies: rust-lang/enzyme#31
This PR can be merged after rust-lang/enzyme#31 has been merged and enzyme submodule has been updated.
I applied diffs on this PR to #151063 then produced artifacts at https://github.com/rust-lang/rust/actions/runs/21091289355
I confirmed that
/path/to/rustc -Zautodiff=Enable -Clto=fat test.rssuccessfully finished when I used those artifacts↑r? @ZuseZ4